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Background 


Since 2011, the Smithsonian Center for Learning and Digital Access (GCLDA) has strived to 
better understand and address the needs of educators utilizing digital assets through a variety 
of research and user testing studies that have led to the creation of a new digital learning 
platform, the Smithsonian Learning Lab (SLL)'. The Smithsonian Learning Lab provides access 
to the digital resources from across the Smithsonian’s 19 museums, 9 major research centers, 
and the National Zoo, to be used as real-world learning experiences. With a repository of over 
1.6 million objects, and a new resource being digitized and added every 6 seconds, the 
Learning Lab provides specialized tools to aid in the discovery and creative use of its rich 
digital materials. For students using the Learning Lab, it is designed to aid in building lasting 
knowledge and critical skills that take learners from simply finding resources to thoughtful 
selection, examination, organization, and creation of new resources. 


The SLL, as it currently exists, was largely informed by the input and practice of diverse and 
effective educators.” The goal of this larger research effort, Understanding the Needs of 
Student Users of Digital Smithsonian Resources, assimilates findings from variety of sources to 
address some key questions around methods or requirements for enhancing student 
motivation and engagement with digital content and tools. 


1. What are the ways that students engage with digital content in academic settings? 

2. What are the motivations for student use of digital content? 

3. What are the interface requirements/scaffolds needed to enable and enhance student 
engagement with rich digital resources? 


Informing the recommendations in this document are findings from the five components of this 
project: 


e A Literature review of research, studies, and reports on the topic of digital learning 
trends and strategies for youth (Vol. 1) 

e Environmental scans of both digital learning environments and social media platforms 
used by youth (Vols. 2-3) 
Analysis of existing usage data for Smithsonian Learning Lab users ages 13-17 (Vol 4.) 
Classroom testing and observations with youth across diverse school environments 
(Vols. 5-6) 


' Milligan, D., and M. Wadman, M. 2015. "From Physical to Digital: Recent Research into the Discovery, 
Analysis . . .” 
<http://mw2015.museumsandtheweb.com/paper/from-physical-to-digital-recent-research-into-the-disc 


overy-analysis-and-use-of-museums-resources-by-classroom-educators-and-students/> 
2 "smithsonian-digital-learning - Teacher Toolkit (Research Findings)." 2013. 28 Sep. 2016 


<https://smithsonian-digital-learning. wikispaces.com/Teacher+ Toolkit+(Research+Findings)> 


Summary of Findings 


The following information represents a compilation of findings from each of the above 
elements. These findings helped the project team to distill a series of features and strategies to 
test with students and determine potential areas for further development to the Smithsonian 
Learning Lab to support the needs of student users. 


Literature Review 


The Literature Review referenced over 100 pieces of research, case studies, reports, and 
journal articles. In reviewing the general trends that emerged from this review, the following 
represent a summary of key findings and strategies that have substantial research support. 


e Learning flow is more consistent and more readily supports achievement when there is 
a high level of correlation or alignment between content, objects/resources, visual 
supports or media, and tasks to aid in persistence and minimize cognitive load.** 

e Developing and sustaining an online learning community focused on inquiry and 
learning is crucial in helping students access both their instructors and peers.° Sharing 
their thinking, their findings, and their learning processes, and having access to those of 
their peers, helps validate work approach, keeps students engaged, and provides an 
opportunity to blend social, cognitive, and teaching dynamics.°® 

e Students’ engagement and performance levels increase when quality content and 
activities are developed by a learner’s own teacher.’ 

e Mutual problem-solving or co-development of learning products helps young students 
make more meaningful connections to their learning and to one another through the 
establishment of relationships focused on learning outcomes.® 

e Presenting students with open-ended, deep, interesting questions and keeping those 
questions central and accessible to students throughout their inquiry process? helps 
guide their progress through complex tasks online.'° 


3 Mayer, Richard E., and Richard B. Anderson. 1992. “The Instructive Animation: Helping Students Build 
Connections Between Words and Pictures in Multimedia Learning.” Journal of Educational Psychology. 
84.4: 444. 

4 Moreno, Roxana, and Richard Mayer. 1999. “Cognitive Principles of Multimedia Learning: The Role of 
Modality and Contiguity.” Journal of Educational Psychology 91.2: 358. 

5 Ito, M. “Living and Learning with New Media - Digital Youth Research.” 2008. 
<http://digitalyouth.ischool.berkeley.edu/files/report/digitalyouth-WhitePaper.pdf> 

8 Higgins, S. 2012. “The Impact of Digital Technology on Learning.” 
<https://v1.educationendowmentfoundation.org.uk/uploads/pdf/The_Impact_of Digital Technologies _o 
n_Learning FULL_REPORT (2012).pdf> 

7 Squire, K. D., et al. 2003. “Designed Curriculum and Local Culture: Acknowledging the Primacy of 
Classroom Culture.” Sci. Ed. 

8 Marks, Ronald B., and Stanley D. Sibley and J. B. Arbaugh. 2005. “A Structural Equation Model of 
Predictors for Effective Online Learning.” Journal of Management Education 29.4: 531-563. 

° Kapa, Esther. 2007. “Transfer from Structured to Open-Ended Problem Solving in a Computerized 
Metacognitive Environment.” Learning and Instruction 17.6: 688-707. 

10 Welch, Wayne W., et al. 1981. “The Role of Inquiry in Science Education: Analysis and 
Recommendations.” Science Education 65.1: 33-50. 


e Having timely feedback on performance," from an instructor or even just in the form of 
external validation of task completion, serves as a key motivator and aids student 
persistence. ' 

e Students bring specific expectations to digitally-supported learning environments, 
including a desire to personally define how accessing and organizing resources and 
information works for them, flexibility in qualifying the expertise of their instructor and 
peers, and the freedom of individual demonstration of knowledge. "? 

e In the area of personal inquiry and progress monitoring,“ visual indicators that 
document a learner’s progress towards completing online tasks and assist in 
monitoring that progress can help young students keep on track, stay motivated, and 
allow them to share their efforts. 


1 Winters, Fielding |., and Jeffrey A. Greene and Claudine M. Costich. 2008. “Self-Regulation of Learning 
Within Computer-Based Learning Environments: A Critical Analysis.” Educational Psychology Review 
20.4: 429-444. 

12 Wolters, Christopher A., and Paul R. Pintrich. 1998. “Contextual Differences in Student Motivation and 
Self-Regulated Learning in Mathematics, English, and Social Studies Classrooms.” /nstructional Science 
26.1-2: 27-47. 

‘3 Chung, Janine, and Felix B. Tan. 2004. “Antecedents of Perceived Playfulness: An Exploratory Study 
on User Acceptance of General Information-Searching Websites.” Information & Management 41.7: 
869-881. 

4 Hattie, John, and Gregory C. R. Yates. 2013. Visible Learning and the Science of How We Learn. New 
York: Routledge. 


Environmental Scans 


Scan of Learning Tools and Platforms: 

The system category receiving the most tools and development effort across all Digital 
Learning platforms reviewed were those directed towards Resource Management with the 
average system addressing 61% of the assessed features. Sub-elements within this category 
included: 


Resource Discovery 

Resource Selection 

Resource Sharing 

Resource Storing/Saving 

Resource Systems Integration/Interoperability 


Subsequently, Learner Management tools and features were also significant with the average 
system addressing 60.9% of the assessed features. 


Scan of Social Media Platforms: 

The system category receiving the most tools and development effort across all Social-Media 
platforms reviewed were those directed towards Community Collaboration Tools with the 
average system addressing 49.6% of the assessed features. Sub-elements within this category 
included: 


Group Tools 

Discussion Tools 
Communication Tools 

User Activity Management Tools 


Classroom Observations and Prototyping 


Student-Use Trends That Were Supported During Observations: 

+ Majority of students made use of digital note-taking tools when available. (Chico cohort) 

+ Majority of students preferred in-system questions and guidance instead of paper-based 
worksheet when both were available. (Chico cohort) 

+ Majority of students were able to set up personal accounts to record information and submit 
to teacher when made available. (Chico cohort) 

+ Data suggested that the overall engagement was higher if activities were based on student- 
or teacher-created SLL collections. (Pittsburgh cohort) 

+ Students’ overall engagement was higher when completing an activity by using different 
means than the SLL (Pittsburgh cohort) 


Student-Use Trends That Went Unsupported During Observations: 

- Majority of students DID NOT opt to work collaboratively when allowed. 

- Majority of students DID NOT take advantage of reviewing peers responses or contributions 
when provided access. 


Student-Use Trends That Were Inconclusive During Observations: 

Students use of progress monitoring for self-tracking and/or as visual indicator to others was 
not effectively tested. Additionally the duration of the tasks were not sufficient to model 
use-cases found in the Literature Review in which students were engaged in projects that 
spanned multiple class periods or multiple weeks of sustained coursework. 


Proposed Technical Development 


The following tables present key findings from four separate instruments that made up the 
Understanding the Needs of Student Users of Digital Smithsonian Resources project. 
Information includes the specific finding, the instrument(s) that supported the finding, and 
proposed development to improve the Learning Lab for young learners. While any one 
instrument might have generated a number of other findings, those reported below represent 
elements that could feasibly be addressed through increased features, new tool development, 
or design enhancements to the platform. 


Summary of findings, features, or strategies to improve Lit. Env. Student User Data 
youth engagement with Learning Lab Review | Scan Testing/Obsv. Analysis 


Finding 1: 

Young learners expect navigation, search and discovery to 

require minimal effort, and a system that can anticipate X X 
search terms and provide quality results or related 

suggestions. 


Proposed Development: 

Enable predictive text that anticipates learner’s search terms. Use predictive text feature to also aid in 
auto-correction of spelling of search terms. 

When resulting resources/collections are limited, provided related resources/collections based on similar 
searches and/or metadata correlations. 

An analysis could be performed to consider if the search filters for both resources and collections should be 
modified to better align with terminology more accessible for young learners. 


Summary of findings, features, or strategies to improve Lit. Env. Student User Data 
youth engagement with Learning Lab Review Scan _ | Testing/Obsv. Analysis 


Finding 2: 

Young learners remain more engaged and persistent in 

completing online learning content when their progress is X 
acknowledged and visual indicators track completion of 

certain benchmarks within the learning content. 


Proposed Development: 

[1 Create an automated in-system tracking layer applied to Learning Collections that provides the learner a visual 
indicator of their progress as incremental steps connected to the number of resources within a given Learning 
Collection or some other related metric. 

{1 An additional approach or feature could be to allow an educator to prescribe their own benchmarks at either 
the Learning Collection or Assignment level to validate learner’s progress in configurations more uniquely tied 
to the learning design as opposed to merely tied numerically to number of resources within a collection.(ie, 
number of questions accomplished) 

*Would be important to tie this feature to assignments so that differentiated benchmarks and progress indicators could 
be applied to different learner audiences as Assignments generated from a single master Learning Collection. 
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Summary of findings, features, or strategies to improve it. Student User Data 
youth engagement with Learning Lab Testing/Obsv. Analysis 


Finding 3: 

Young learners benefit from receiving regular feedback on 
tasks and contributions they complete within their online 
learning environments from their educators, peers, or other 
members of the learning community. 


Proposed Development: 

L1 Develop a feature that allows educators or designers to create follow-up responses to given questions that will 
automatically appear to learners upon submission of their own answers. Responses can obviously be 
additional guidance for wrong answers on T/F or Multiple Choice or acknowledgement of correct answers. For 
open response questions, feedback can take the form of an expected response or key items/concepts the 
educator was hoping to capture so the learner can self-reflect on proximity to the desired response. 

Create a feature that ties into progress monitoring* that allows an educator or designer to create and include 
feedback to a learner when various completion points or benchmarks are reached within the learning 
collection. 

1) Provide a feature that allows educators to comment upon given learner responses to questions when 
reviewing and a means for learners to receive those comments. 
*Can be constructed as an aligned element to Progress Monitoring solutions identified in Finding 2 above. 


Summary of findings, features, or strategies to improve Lit. Env. Student User Data 
youth engagement with Learning Lab Review Scan _ | Testing/Obsv. Analysis 


Finding 4: 

Young learners benefit when provided the means to X X X 
annotate or record notes that document their thinking, 

findings, and questions when engaging learning content. 


Proposed Development: 

1 Develop a feature that provides learners a persistent note-taking utility that can carry guiding questions posed 
by the learning collection or teacher, provides a space to type or copy content related to their 
thinking/learning/discoveries while accessing multiple resources both within and beyond the Learning Lab 
(externally linked resources). 

Preferably, learners could elect to make all or parts of their documentation private or shared to peers, 
community, or teacher. 


Summary of findings, features, or strategies to improve Student 
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youth engagement with Learning Lab Testing/Obsv. 


Finding 5: 

Young learners, given opportunities to engage their peers in 

collaborative discovery and shared findings, are more X X 
engaged in extended reflection on their own thinking and 

conclusions. 


Proposed Development: 

(1 Develop features that allow learners to select peers as collaborative partners for note-taking, collection 
assembly and authoring, or in answering questions and completing assignments. 
Develop features that allow an educator or designer to optionally activate questions that can be addressed by 
teams of students. 
Develop a feature that allows an educator or designer the option of exposing learners to the responses of their 
peers to a given question before or after submitting their own response. 


Summary of findings, features, or strategies to improve Lit. Env. Student User Data 
youth engagement with Learning Lab Review Scan _ | Testing/Obsv. Analysis 


Finding 6: 

Young learners can more readily engage in self-regulated 

learning and show increased performance on tasks when X X 
content, tools, and inquiry are closely aligned to minimize 

transactional distance. 


Proposed Development: 

[J While the Learning Lab has taken initial design steps to align guiding questions and annotations visually with 
the primary learning resources, additional analysis should be performed to look for further improvements. 

1 Consider solutions that would allow learners persistent access to guiding questions and the means to respond 
or note-take when visiting external resources. Tools that travel with a learner when accessing an external 
resource, or that allow a learner or educator to pull key content or media from an external resources back into 
the digital learning environment could be considered. 
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